1. Field of the Invention
This invention relates to functional verification of a design. More particularly, this invention relates to systems and methods for the efficient creation of test cases in order to verify combinations of several different functions of a hardware or software design.
2. Description of the Related Art
Functional verification is widely acknowledged to be a bottleneck in a hardware system's design cycle. Indeed, up to 70% of development time and resources are typically spent on functional verification. Allowing users to find design flaws, and fixing them in a subsequent release would be unwise and costly for three main reasons: (1) harm to reputation and brand-name; (2) a high cost of recall and replacement when there is a large installed base; and (3) litigation, should design flaws cause injury.
In current industrial practice, dynamic verification is the main functional verification technique for large and complex systems. Dynamic verification is accomplished by generating a large number of tests using random test generators, simulating the tests on the design-under-verification, and checking that the design-under-verification behaves according to its specification.
The rationale behind verification by simulation is that one acquires confidence in the correctness of a design-under-verification by running a set of test cases that encompass a sufficiently large number of different cases, which in some sense is assumed to be a representative sample of the full space of possible cases. The ability of the design-under-verification to correctly handle all cases is inferred from the correct handling of the cases actually tested. This approach is discussed, for example, in the document User Defined Coverage—A Tool Supported Methodology for Design Verification, Raanan Grinwald, Eran Harel, Michael Orgad, Shmuel Ur, and Avi Ziv, Proc. 38th Design Automation Conference (DAC38), pages 158-163, 1998. When conducting simulations, it is desirable to define a particular subspace, which is considered to be “interesting” in terms of verification, and then to generate tests selected at random that cover the subspace.
Test cases developed by algorithms such as the foregoing are typically implemented on a test generator, which may optionally bias the tests based on internal testing knowledge. Such test generators are described in the following documents: Model-Based Test Generation For Processor Design Verification, Y. Lichtenstein, Y. Malka and A. Aharon, Innovative Applications of Artificial Intelligence (IAAI), AAAI Press, 1994; Constraint Satisfaction for Test Program Generation, L. Fournier, D. Lewin, M. Levinger, E. Roytman and Gil Shurek, Int. Phoenix Conference on Computers and Communications, March 1995; and Test Program Generation for Functional Verification of PowerPC Processors in IBM, A. Aharon, D. Goodman, M. Levinger, Y. Lichtenstein, Y. Malka, C. Metzger, M. Molcho and G. Shurek, 32nd Design Automation Conference, San Francisco, June 1995, pp. 279-285.
The term coverage concerns checking and showing that testing has been thorough. Coverage is the prime measurement for the quality of a set of test cases. Simply stated, the idea in coverage is to create, in a systematic fashion, a large and comprehensive list of tasks, and to check that each task is executed in the testing phase. Ultimately, higher coverage implies greater chances of exposing a design flaw.
In complex systems, obtaining coverage that involves interaction among different functions is a particularly difficult task, as the interactions of different functions within the design can be non-determinative and difficult to reproduce.